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INTRODUCTION 



The University of Maryland and other institutions have cooper- 
ated with the Office of Education and State Departments of Education 
in conducting seminars in the areas of Leadership Development and 
Program Planning Budgeting and Evaluation designed to improve and 
strengthen the nation^ s programs of vocational and technical educa- 
tion. It is believed that these efforts have had a significant impact 
upon the vocational and technical education programs offered by the 

various states . 



The need is great for a quantity of high quality leaders. It is 
clear that vocational and technical education personnel are faced with 
challenges which will not diminish in the foreseeable future and 
which cannot be ignored or taken lightly. It is appropriate to pause 
and to weigh the strengths and weaknesses of our experiences gained 
in conducting the various seminars and to suggest possible improve- 
ments in these activities which should result in a strengthened 

program . 



The participant-consultants sharing in this evaluation conference 
represent and reflect the leadership-action level of the vocational 
and technical education personnel across the nation. Within ™ 
limits of time it was planned that the objectives j the content, the 
agenda, and the techniques used in the various seminars would be 
considered through the experience and insight of the participant- ^ 
consultants. It was hoped that a formulation of functional guidelines 
for future seminar series would result. 



The University of Maryland and the Office of Education regarded 
this conference as a most meaningful activity. It should contribute 
to the development of the nation »s leadership in this dimension of 

education. The knowledge and insight ^ 

improved program planning and effective budgeting and evaluation 
techniques should^result in sufficient progress to place our programs 
on a more valid basis upon which we can look to the future with 
increasing confidence. 



Clodus R. Smith 
November, 1967 
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SUMMARY 



This final report presents the evaluation 
study of the 1967 Leadership Development Seminars 
and the Program Planning and Budgeting seminars 
held at the University of Maryland jand the 
University of California at Los Angeles • 

The participant-consultants represented the 
action level of the State Departments of Education 
and the Office of Education concerned with voca- 
tional-technical education. 

Strengths and weaknesses of the 1967 seminars 
were studied so that suggestions for future 
seminars could be developed . 

Evaluative approaches to the problems were 
considered. Effective techniques of organizing 
and conducting these group meetings were discussed. 



Objectives and procedures were related to the 
current challenges in the field of vocational- 
technical education. 



REMARKS RELATIVE TO THE 1968 SEMINARS 
Dr. Duane Nielsen, U.S. Office of Education 



The evaluation seminar is an important phase of the overall 
activity of the IDS and the PPBS meetings. 

The most important aspects of the seminars include; 



Who is There? 

The selection procedures should be closely related to the objec- 
tives of the seminar so that as far as possible, the people who are 
in attenviance can profit from the seminar and multiply the effects or 
the activity within their area of responsibility and educational 

leadership. 

What is Accomplished? 

The agenda of the seminar should reflect careful planning. The 
most capable resource people should be involved and the techniques 
employed should be effective in relation to the goals for which the 
seminars are planned. 



How It is Accomplished? 

Are the barriers to effective communication between individuals 
and groups overcome so that the participants are moti^ted to signi- 
ficant action in the period following the seminars? The techniques 
should reflect that progress which has been made in communication, 

and in group dynamics. 



The Nature of the Final Report 

This should Include some reflections of the impact attained by 
the seminar as well as the bare recital of the 

Some basic simplified criteria are needed to jell out the essence of 
the seminar wittin the limits of practicality. There sh^d ^ 
presentation of VJhat Happened of Value at the . 

a bit more effort and ingenuity, but it will enhance i, 
report to the USOE and to the Voc-Tech field, 

A covering letter has been prepared which would have been sent 
out to thrSeld early in 1967 if possible to provide more generous 

lead time in preparation of proposals . J;", of'^the 

interfered with that time table. However, the general mailing ot t 
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prospectus will be done very soon so that all interested agencies and 
institutions can develop and submit their proposals for the 1968 
seminars . 

The guidelines will probably be the same or similar to those for 
1967. The proposals should indicate all components and activities to 
be funded. The deadline will probably be 15 January 1968. 

Fifteen priority areas have been identified for the 1968 seminars 
of which Leadership Development and PPBS are near the top of the list. 

Present plans contemplate two seminars in IDS with thirty parti- 
cipants in each for a one week period. 

The PPBS plans contemplate two seminars of forty participants 
for a two week period. 

In addition there are seminars contemplated for the study of 
vocational education planning and development for large ci^ systems; 
manpower survey techniques, and innovative curriculum developments, 

among others. 
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RATIONALE AND RETROSPECT 



N. Edwin Crawford 

LDS and PPBS Evaluation Conference 
University of Maryland, November 26, 1967 



Vocational education has made considerable progress in such areas 
as enrollments, funding, facilities and research. However, some pro- 
blems still confronting all levels of vocational education in our 
nation include: big cities, rural poverty, educational programs for 

persons with special needs, labor mobility, critical manpower short- 
ages, and high school dropouts. 



The ccope, urgency, and complexity of problems such as these 
demand the utilization of different concepts, tools, and procedures 
than some of the obviously ineffective ones currently in use. Educa- 
tors and non-educators from various segments' of our society are 
calling for changes in public education, including the vocational 
establishment . 

Of the various available alternatives for effecting desired 
changes, developing and implementing a Planning, Programming, and 
Budgeting System (PPBS) for vocational education offers the highest 
potential for immediate payoffs. Of the various weaknesses of present 
approaches, perhaps the major ones are the failures to work toward 
specifically stated, valid objectives; to consider a wide range of 
alternate programs for accomplishing objectives; and to provide 
realistic control and evaluation tools and procedures. 



Herein, then, lies the plea for the adoption of PPBS in voca- 
tional education, for the antithesis of our present failures is the 
essence of PPBS concept and procedures. In short, PPBS is concerned 
with relating the flow of dollars to the accomplishment of program 
objectives, and with weighing rationally alternate ways of doing 

things . 



Within this framework, "program” is an integrated 
Programming-Budgeting "package." ’^Planning" is conceded with the 
identification of a range of possibilities for selecting courses of 
action through a systematic consideration of alternatives. Trogram- 
ming" involves the specific determination of people, instructional 
materials and equipment and facilities necessary for accomplishing 
program objectives. And, "Budgeting" is the translation of a plan of 
activities into specific materials, personnel, time and cost factors. 



"System," as related to this concept of PPBS, is a set of Plan- 
ning, Programming, Budgeting elements, organized to accomplish 
optimum program results in terms of specifically stated and measurable 
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objectives. A system is based on an analysis that assigns and defines 
the respective roles to be played by the various resources of the 
system, as well as the inter-relationships involved. Primarily, the 
system is concerned with objective setting, planning, quality control, 
and efficiency. 

PPBS has been used successfully in the Defense Department and 
several other government agencies, as well as the private sector for 
the past six or seven years. However, this system has been only 
recently applied to education. Results of PPBS efforts to date have 
been minimal, but there are indications of increased Federal, State, 
and local interest and involvement — at least in certain aspects, 
such as "Systematic Program Planning." Much progress was made in this 
respect at the National Seminar on Program Planning, Budgeting, and 
Evaluation held at Maryland University during the summer of 1967. 
Several national PPBS Seminars are planned for the coming year. 

The true test of a system, including PPBS, of course, is the 
extent to which it permits the successful attainment of the objectives 
for which it was designed, and the efficiency with which it operatives. 
Although it has proved successful in various non-educational organi- 
zations and institutions, PPBS must still be tested for its efficacy 
in education. 

Perhaps certain questions are in order: 

1. Is PPBS as now known both feasible and practical? 

2. Does education have a "substitute" system? 

3. Might we modify the Defense Department’s system, and other 

versions, to meet our needs? 

4. Could we develop a new system or approach to planning, 

programming, and budgeting to meet our needs? 

5. What alternatives does vocational education have for 

learning about and developing leadership in PPBS? 

6. What are the major inhibiting factors in establishing PPBS 

at the national, regional, state, and local levels? 

7. What are our alternatives for circumventing or overcoming 

these inhibiting factors? 

8. What are the characteristics of the general roles and 

specific responsibilities regarding the development and 

implementation of PPBS at all levels of the vocational 

education establishment? 

9. Is vocational education significantly different from busi- 

ness, industry, government, and other areas of education — 

with regard to PPBS? 
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EVALUATION OF LEADERSHIP DEVELOPMENT AND PPBS SEMINARS 

A. H. Krebs 



The evaluations of the seminars was conducted in terms of the 
general objectives of the seminars. Both process (conduct of the 
seminars) and product (changes in participants) were evaluated. 

In addition to personal observation of the seminars, five 
different instruments were used in the evaluation. A ’’reaction 
instrument” was used to check on the response of the participants 
as the seminars were in process. End-of-seminar evaluation instru- 
ments were administered to the Leadership Development Seminars: a 

conference leading technique inventory. 



Summary of Leadership. Development Seminar Evaluation 

The seminars must be judged as highly successful. All instru- 
ments and observations indicated definite growth on the part of the 
participants. Rated highly were the conduct of the conference, the 
resource person presentations, degree of participation, conference 
techniques used, exchange of ideas, and group interaction. ^ ^ ^ 

positively rated, the task force assignment was not rated as highly 

as the other items. 

Suggestions for improvement must be considered in the light of 
the overall favorable reaction. The following suggestions should be 

considered: 



1 . 

2 . 

3 . 

4 . 

5 . 

6 . 

7 . 



More freedom in eating arrangements. 

A one-week conference. _ 

A more practical task- force problem. 

More detailed pre-seminar communication regarding the 



program . 

Increased involvement 
Fewer resource person’s 



of participants. 

and having them available for more 



of the seminars. 

Selection of participants 



in terms of seminar objectives. 



Summary of PPBS Seminar Evaluation 

The seminar was successful in bringing about participant growth 
and in providing guidelines for planning future seminars. 

Some unrest was revealed by observation and by the evaluation 
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instruments. This could have been due to: 

1. Unwillingness of some participants to live up to an agreement 
made when the invitation to attend was accepted. 

2. The timing of the seminar when state legislatures were deal- 
ing with vocational- technical education legislation. 

3. An incomplete understanding by participants of the objectives 
for the seminar. 

4. The sequencing of some parts of the seminar. Some presenta- 
tions should have been scheduled earlier: the group task 

assignment should have been made earlier. 

Suggestions, in addition to those above, included: 

1. Inviting state personnel as teams made up of persons with 
PPBS responsibilities. 

2. Maintain the national emphasis through using the USOE 
personnel. 

3. Provide a program broadly oriented, rather than focusing on 
individual states, and relate program to state problems. 

4. Develop and use a PPBS task inventory to use in evaluation 
and to serve as one way to orient participants to the program. 

5. Develop a set of data to use as a problem to be tackled by 
groups in ’’competition" with each other. Prior decisions^ 
made by USOE personnel could be used as criteria for judging 
team success . 



1. Seminars should be so scheduled as to provide time to per- 
form assigned tasks. ^ ^ 

2. Seminar staff should be aggressive and positive in identi- 
fying and resolving differences in philosophy, definitions, 
and facts as presented by various resource persons. 

3 . Resource persons should be invited to be present for at 
least a substantial part of the seminar. 

4. A continuing emphasis’ should be maintained on the relation- 
ship between the seminar objectives and program. 



General Comments 
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TEACHING-LEARNING PROCESS CRITERIA 
FOR CONTINUING EVALUATION 

Einar R. Ryden 

LDS and PPBS Evaluation Conference 
University of Maryland, November 26, 1967 



A teaching - learning - evaluation system may be analyzed into 
six components. The interaction of these components and the applica- 
tion of appropriate criteria can yield a functional evaluation system. 



A fundamental component can be designated as Data. There is 
always the current situation from which we are to move in a desired 
learning direction. An inventory of data is essential so that 

you can be aware of what can be assumed as basic knowledge, skills 

or attitudes. 



fl 

Based on these data, the determination of what is to be taught 
can be made. Again, the criteria should be related to 
goals so that function and relevance can prevail. In addition, 
is the question of who is to benefit from the teaching process. The 
nature and quality of the learners may limit or direct the data to be 
included in the teaching - learning situation. 



The second component may be designated as the program itself. 

Here we must establish a detailed list of tasks to be perfomed. In 
vocational education we need to consider tasks that . 

on the job or in a simulated job situation, as opposed to those that 
shoSd he a part of the »»formal»» education process. Task dete^intd 
objectives can be formulated which can be integrated into 
objectives. It may be that the nature and quality of the instructors 
available to the programmer may be a conditioning factor. 

A third component receives the designation of METHNICS, which is 
a term I have coined to cover methods, techniques and devices included 
in actual teaching process, Here «^asic question is how to teach 
effectively to attain the objectives. The criteria should reflect 
functional efficiency as well as availability of means. 

The next component has been labeled performance. Here we strive 
to have the learners do the tasks included in the propam. We hope 
tLfwe Save established exactly what it is we want the learner to be 
able to do at the end of the instructioial 

do at the start. In the field of vocational education we can often 
describe the operational skill requirements considerable ac- 

curacv. Ii is often more difficult to measure the degree of under 
standing accompanying the skill patterns . Some learners can perform 
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by rote, which is fine unless some part of the requirement changes 
somewhat* Then his performance can break down unless he has the 
degree of understanding to adjust to the change. 



In the next component, we involve the processes of review and 
adjustment. Here we consider the feedback from phase four - perform- 
ence and make such changes in METHNICS as are obviously apparent. 
Sometimes we need to re-examine the rationale we have set up for the 
program to determine if our previous analysis still appears sound. 
Perhaps some of our objectives are not attainable and need to be 
restated in more functional terms. Also, it may be that some of the 
criteria we have employed need to be reviewed, or others developed. 



If we consider these five components as gears in the machine 
concept, then as the center gear meshing with the other five gears, it 
is the component we may term evaluation. It can relate one component 
to another as progress is measured and compared. In « ^ 

a catalytic agent which helps, give meaning to all of the components 
in their cooperative effort in the teaching - learning process. 
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EVALUATION CONFERENCE 

LEADERSHIP DEVELOPMENT SEMINAR AND PROGRAM PLANNING, 
BUDGETING AND EVALUATION SEMINAR 



Center of Adult Education 
University of Maryland 
November 26 - 28, 1967 



SUNDAY, NOVEMBER 26, 1967 
REGISTRATION 



1:30 P.M. - 2:30 P.M. 



Lobby, Center of 
Adult Education 



OPENING GENERAL SESSION 



2:30 P.M. - 4:45 P.M. Room C 

Greetings 

Clodus R. Smith 
Sherrill D. McMillen 

Introductions 

Erna Chapman 

Purposes of Conference 

Clodus R. Smith 

A Concept for Evaluating Educational Programs 
Einar R. Ryden 

Evaluation of Leadership Development and PPBS Seminars 
Alfred H. Krebs 

4 

Program Planning and Budgeting Seminars: Rationale and 

Retrospect 

Edwin Crawford 

5:30 P.M. - 7:30 P.M. DINNER Holiday Inn 



10 






SESSION II - SUNDAY, NOVEMBER 26, 1967 



8:00 P.M. - 9:00 P.M. ^ 

Group and individual assignniGn'ts: 

Erna Chapman 

Group A - Evaluation of Leadership Development Seminar 
Series ® 



Ted Cote - Chairman; John Thompson - Recorder 

1. Identification of strengths of the Leadership 
Development Seminars 

a , Objectives 

b , Program content 

c , Techniques 

d , Resoyrce persons 

e , Location 

2. Identification of weaknesses of the Leadership 
Development Seminar series 

a . Objectives 

b . Program content 

c . Techniques 

d . Resource persons 

e . Location 

3. Suggestions for future seminars 

Group B - Evaluation of the Program Planning and Budgeting 
System Seminar 

Glenn Strain - Chairman; John Connolly - Recorder 

1. Identification of strengths of the Program Plan- 
ning and Budgeting System Seminar 

a , Objectives 

b , Program content 

c , Techniques 

d , Resource persons 

e , Loca tion 

2, Identification of the weaknesses of the Program 
Planning and Budgeting System Seminar 

a . Objectives 

b . Program content 

c . Techniques 

d . Resource persons 

e. Location 
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3. Suggestions for future seminars 



MONDAY, NOVEMBER 27, 1967 
SESS ION III 

Groups A and B meet separately 



9:00 A.M. - 10:15 A.M. 



Group A 
Group B 

Planning future seminars 

1. Defining objectives 

2. Determining appropriate topics 

10:15 A.M. - 10:30 A.M. coffee break 
12:00 Noon 

SESSION IV 



Room D 
Room C 



Coffee Shop 



Groups A and B meet separately 



1:15 P.M. - 3:00 P.M. 

Group A 
Group B 



Room D 
Room C 



Structuring future seminars 

1. Sequence of topics - priorities, day, time, etc. 

2. Personnel suggestions - consultants and resource 

persons - related to topics . ^ 

3. Selection and assignment of conference techniques. 



3i00 P.M. - 3:15 P.M. coffee break 



Coffee Shop 



3:15 P.M, - 5:00 P.M. Continuation of Session IV 



12 



o 



TUESDAY, NOVEMBER 28, 1967 
SESSION V 



8:45 A.M. 



General Session - Groups A and B 



10:15 A.M. 



Room C 



Review of Progress 

1. Comparison of 1967 seminars with plans for future 

seminars „ . . 

2. Refinement of plans for future seminars 



.0:15 A.M. - 10:30 A.M. 



Coffee Shop 



coffee break 
SESSION VI 

General Session - Groups A and B 

.0:30 A.M. - 12:00 Noon ^ 

Tiichard Nelson - Chairman; Joe Malinski - Recorder 



1. Administrative procedures 

a. Selection procedures 

b. Housing 

c . Transportation 

d. Food Service 

e . Secretarial and duplication services 

f. Other services and arrangements 

2. Administration of projects 

Duane Nielsen 



3. Review of evaluation process 
Einar R. Ryden 



4 . Conference summary 
Clodus R . Smith 



2:00 P.M. 



adjourned 
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EVALUATION OF lEADERSHIP DEVELOPMENT SEMINARS 
Group A Ted Cote - Chairman John F. Thompson - Recorder 



Evaluation of Leadership Development Seminars on the national and 
state level in vocational-technical education provides the most effec- 
tivrmeans for continuous growth in this field. Here the leaders 
become aware of the strengths and weaknesses of the overall program. 

The ongoing programs must be studied to insure their continued 
usefulness in tL world of work. Updating tools and ™“hine^ in a 
narticular ioh-training area are sometimes as necessary as changing 
skills. The resource persons connected with ongoing programs 
vSiabie ^ntributions to each other in their knowledge of h^ their 
part of the country affects and is affected by laws, skill demands, 
objectives j and attitudes. 

Basic concepts of vocational-technical education must be reviewed 
in or^r to create new objectives. The U.S. Office of ^as ^ 

oredicted that by 1975 six million secondary students, 1,250,000 pos. 
S schLl studLts, and 6,500,000 students in 

be enrolled in some form of occupational training, with a substant^l 
uLberof "special needs" classes. Also, there has been a prediction 

?hat Lch em^loyea person in his early «^l^^i/S"that 

least eight or ten times during his working years. ^ 

bringing together resource people will be a means . 

ideS concerning vocational-technical educarion. f 
meeting these new ideas can be developed and passed on through 
proper channels to become effective in a short period of time. 

Seminars to develop leadership for the field of vocatioiral- 
technLal education should have realistic and functional goa ,s. These 
may be shown schematically in this manner; 



THE GOAL 

To: Build 

Improve 
CxirsTicl 

Develop An: Awareness ^ 

Appreciation 

Understanding 

Of the appropriate: Role 

Function 

Place 

Of the: People 

Structures 

Laws 

Trends 

Developments 
Objectives 
14 Attitudes 



which will contribute to continued effective growth of vocational- 
technical education in the United States. 



Summary of Objectives for Leadership Development Seminars 

1. As a result of the seminar the participants should be able to: 

a. Describe administration procedures and practices in 

Vocational Education. 

b. Identify recent trends and developments affecting Vocational 

Education and locate additional informational sources. 

c. Explain the legislative process and its relationship to 

Vocational Education. 

d. Apply evaluative processes to Vocational Education. 

e. Identify current socio-economic pressures and their inter- 

actions with Vocational Education. 

2. As a result of the seminar the participants should have a 

working knowledge of procedures for: 

a. Organizing and conducting group meetings. 

b. Conducting small group discussions. 

c. Identifying and appraising program needs. 

d. Matching program with legislative intent, 

e. Identifying and selecting information for program 

improvement . 

f. Using research reports. 

g. Interpreting Vocational Education to the public. 
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AGENDA TOPICS - GROUP A 
Leadership Development Seminar 1968 



Session #1 

Topic - Orientation 

Resource Person - Stress the role of a leader and the techniques 
to watch for in the conference. 

Technique - 



Session #2 

Topic - What do interested groups expect from Vocational 

Education hours; 

Resource Person - 

Technique - Panel of emplqxers representative of Vocational 
Education fields 



Session #3 

Topic ” What’s new in Vocational Education Environment (3 hours) 
Resource Person -- 

Technique - Speaker and listening team with stress on the application 
of research findings to their program 



Session #4 

Topic - New ideas on Vocational Development of Youth (1^ hours) 
Resource Person - 

Technique - Speaker and reaction panel on the implications for 
Vocational Education 
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Session #5 & 6 

Topic - State plans and projected activities (3 hours) 

Resource Person - 

Technique - Symposium - Federal, regional, state, and local person 
speaking on the significance of the program question 
and answer period 



Session #7 & 8 

Topic - Socio-economic factors affecting Vocational Education 

Vocational Education factors affecting socio-economic 
situation (3 hours) 

Resource Person - 

Technique - Symposium of a Vocational Educator and a Sociologist 
or economist followed by small group discussion and 
reports. Report to be on Friday morning, could extend 
itself to the afternoon 



Session #9 

Topic - Contributions of other agencies to Vocational Education 
effort (2 hours) 

Resource Person - 

Technique - Two speakers; Industry and Education 
Buzz Session 



Session #10 & 11 

Topic - How Vocational Education law is developed and implemented 
including the role of organization affecting current 
legislation hours) 

Resource Person - 

Technique - Symposium; Legislator, AVA & a state director 
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Session #12 

Topic - Developing leadership skills at the State level (I'g hours) 
Resource Person - 

Technique - Problem Clinic: State leaders to react to pre*-prepared 

problems by participants 

Session #13 

Topic - Techniques in planning and organizing group meetings 
^ (1^ hours) 

Resource Person - 

Technique - presentation of topics 

m 

Session #14 

Topic - Program planning 
Resource Person - 

Technique - Case study - Materials to be prepared in advance 

and issued to participants at end of day 2 or day 3. 
Session for overnight study and report next day 

Session #15 

Topic - Responsibility of levels of Leadership (3 hours) 

Resource Person ~ 

Technique - Symposium: Federal? regional? state? & local 

Session #16 

Topic - Public relations for Vocational Education (1'2 hours) 

Resource Person - 

Technique - Panel of participants 
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Session #17 

Topic “ Progress report on National Advisory Council (1^ hours) 
Resource Person - 

Technique - Presentation: Questions and answers 



Session #18 

Topic - What does the evaluation process mean to 



Resource Person - 

Technique - Symposium: State, private and local, Professional 



Session #19 
Topic - Evaluation 
Resource Person - 
Techrkique - 



(Time as 
needed and 
available 
Perhaps 
report at 
or after 
luncheon) 
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PROGRAM PLANNING, BUDGETING AND EVALUATION SEMINARS 
Group B Glenn Strain - Chairman John Connelly - Recorder 

Objectives 



It was agreed that the three objectives listed below (l>2j^3) 
might be included within the general meaning and scope of the fourth 

objective listed. 

1. The development of insights into the principles and process 
of program planning, budgeting and evaluation; 

2 To stress the importance of systematic planning and develop- 
ment to meet constantly changing requirements of vocational- 

technical education; , ^ 

3. To involve State, Regional, and Headquarters staff members 
in learning experiences necessary to design improved program 
plans for vocational education; 

4 To develop a cadre of vocational educators knowledgeable i 
systematic program planning, budgeting and evaluation. 

It was the consepsus that objective 4 should be the prime objec- 
tive of the seminar. 



Special Item - Participants 

It was the consensus that selection of personnel for the seminars 
should be geared to those persons having decision making responsib 
imes in SrvLtional-tLhnical area, or closely related areas. 
Examples* - State Directors, Assistant Directors, Program plann , 
RCU Lordinators, Fiscal and Administrative officers. Teacher 

Educators . 

participants should agree to attend the entire seminar. 



Program 



1 . 



2 . 

3. 



ntent Comments 

everal presentations were thought to be irrelevant to the 
biectives although containing good information. 
hfprLentations should focus directly on the objectives, 
here should be some required reading prior to attendance at 
he seminar. This material should afford an insight into 
bl S program and establish the need for the seminar. 

i sWd familiarise the participants with the fomat of 

:he seminar and designated materials from his state that 
rould be helpful contributions^ 



Techniques 

These were considered to be generally of good nature and quality. 
The following suggestions are offered: 

1. There should be earlier involvement of the group in model 
development. 

2. There should be more in-depth treatment of the budgeting 
and evaluation phases of the system. 



Resource Personnel 

It was suggested that 

1. Personnel should be available to the workshop activities of 
the groups as resource persons j as opposed to a one-shot, 
hit and run, arrangement. 

2. It may be more effective to have three or four experts, who 
have used the system, to be available throughout the entire 

seminar. .n. j 

3. Resource personnel should be selected and notified as early 

as possible to insure availability and preparation. 



AGENDA T0P7XS - GROUP B - PPBS 



First Day 

Session #1 (9:00 

Chairman: Comments on issues of the day by Dr. Venn prior to his 

introduction of Dr. Hitch. 

Topic: Objective of keynote speech is developing and emphasizing 

need for PPBS* 

Session #2 (10=30 A.M. - 12:15 P.M.) 

Topic: Relate the need to our fields of xnterest and responsibxlity . 

Resource: Should consist of’ federal, state, and regional personnel. 

Technique: Should be determined by the proposed chairman of the 

panel . 

Session #3 (1=30 P-”- ' 2=00 P.M.) 

Topic: Orientation to this seminar 

Resource : Planning Committee 

Technique: Optional 

Session #4 

Topic: Deals with history, concept and development of PPBS. 

Resource: Joe McGivney 

Technique: Optional 
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Second Day 



Session #5 (9=0° ' ^“=30 A.M.) 

Topic; Establishing of objectives* Within thisj deal with 
analyzing? implementing? and evaluating* 

Resource: Peter Pipe (Joe Malinski? Chairman) 

Technique ; Optional 



Session #6 (10=45 A.M. 

Topic; Macro-Economics 

Resource: New and additional resource personnel* 

Technique; Lecture 



12:15 P*M*) 



Session #7 

Topic; Macro-Economics 

Resource; New and additional resource personnel* 
Technique; Lecture 



3:00 P.M*) 



Session #8 

Topic; Cost/Benefit 

Resource: Jacob Kaufman? Penn State 

Technique : Optional 



Third Day 

(9:00 A.M. - 12:15 P.M.) 
with break 

Tonic; Aimed at identifying gaps, locating them and relating tools 
and processes that are available to this process* 



Session #9 



Resource; Office of Education personnel > 



Technique; Optional 
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Session #10 (1=°° " 4=30 P.M.) 

Topic: State PPBS considerations (A case study). 

Resource: Sherrill McMillen 

T echnique : Opt ional 



Session #11 

Topic: Orientation of planning teams 

Resource : In house 

Technique: Optional 



(9:00 A.M. - 10:30 A.M.) 



Session #12 ' (^-<> = 45 A.M. - 12:15 P.M.) 

Topic : Sources and uses of Data 

Resource; Department of Labor, Sam Burt, etc.; Bureau of Census, 
Researchers; Office of Education, etc.; Camps" 

Technique : Optional 



Session #13 

Topic: Organization of groups 

Resource: In house personnel 

Technique: Optional 



(1:30 P.M. - 3:00 P.M.) 



Session #14 



(3:15 P.M. - 4:30 P.M.) 



Topic: Review of a case study (Ex. Dr. Legg’s Health Program, 

DT.c?sect it for objectives) 



Resource: In house personnel 

Technique: Optional 



24 



o 



Fifth Day 



(9:00 A.M. - 12:15 P.M.) 



Session #15 

Topic: Techniques of estimating or predicting 

Resource: Additional resource people 

Technique; Optional 

Session #16 (1:30 P.M. - 4:30 P.M.) 

Topic: Work problems applied to techniques of predicting 

Resource: Joe McGivney and Joe Malinski 

Technique: Individual exercise in technique versus group workshop. 

Problems to be assigned by resource persons. 

Sixth Day 

Session #17 (9:00 A.M. - 10:30 A.M.) 

Topic: How decisions are made presently in educational planning. 

How they may be made with PPBS. 

Resource: To be determined 

T echnique : Optional 



Session #18 ^,10:45 A.M. 

Topic: Orientation of groups to assignment 

Resource; In house personnel 
Technique: Present model and discuss 



12:15 P.M.) 



Session #19 



(1:30 P.M. - 4:30 P.M.) 



Tonic: This should be a learning experience. The group should be 

involved in a step by it ep application of a model. During 
this process? constant feedback? evaluation and modification 

as indicated. 



Resource; In house personnel 
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Technique: Optional 



Seventh Day 

Session ^|20 (9:00 A»M» - 12:15 P»M*) 

with break 

Topic: Planning units to deal with first problem 

Resource: In house personnel 

Technique: Group Work 



Session ^21 (1:30 P.M. - 3:00 P.M.) 

Continuation of previous session* Final part of this session should 
be a report and evaluation of the work on this problem. 



Session #22 (3:0° “ 4:30 P.M.) 

Topic: Planning units to deal with second problem 

Resource'. In ]iDuse personnel 
Technique: Group work 



Eighth Day 



Session ^23 (9:00 A.M. - 

Continuation of session #22 

Session #24 (1:3° P-M. - 

Topic: Report and evaluate units work on second problem 

Resource: In house personnel 

Technique; 



12:15 P.M.) 
4:30 P.M.) 



Ninth Day 



Session #25 



(9:00 A.M. - 12:15 P.M.) 



Topic; Projected Activities report review. (P.A. seen as a state 
tool and a federal requirement) Relating it to "Camps”-- 
Coooerative Area Manpo wer Planning System * 



Rgsouitcg! In Tiousg pGirsonnGl snd othGr’S "to og dGliGmiinGd* 

TGchniquG: As appropriate 

Session tf26 (1:30 P*M. - 4:30 P.M.) 

Topic: Reviev? and evaluate ’^Annual Descriptive Report j” relating it 

to *T>rojected Activities Report.” 

Resource: In house personnel 

Technique: - - 

Tenth Day 

Session #27 (9:00 A,M. - 12:00 Noon) 

m 

Topic: Interrelationships of reports and plans. Showing how they 

collectively constitute a system. 

Resource: In house and other personnel 

Technique: Optional 

Session #28 (12:30 P.M. - 2:00 P.M.) 

Topic: Lunch and summary of seminar. 

Resource: Dr. Smith and others 

Some Suggested Resource Personnel 
Jacob Kaufman - Penn State 

Leonard Lecht - National Planning Associations Washington? D.C. 

Joe Duncan - Battelle Memorial Foundation? Columbus? Ohio 
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GENERAL SESSION CRITIQUE 



Richard Nelson - Chairman; Joe Malinski - Recorder 



A number of areas must be considered in order to 
develop successful procedures for Leadership Seminars, ^e attache 
»Guide for Planning" may be helpful. The discussion evoked the 

following comments. 

Selection of Participants 

Criteria should be developed for selecting )^rtici|^nts witt^^ 
aoDronriate spread of experience and responsibility. Identity the 
training objectives and relate selection to them. The guidelines foom 
the office of Duane Nielsen will indicate the suggested groups 
varioS sLinars. The proposals should stipulate the ^pes and pre- 
requisites of participants that will be set up and followed. 

Housing 

There may be advantages in the housing of all participants in one 
facility that offset those of dispersed housing at individual option. 
The arSngements should be clearly stated so that participants know 

the requirements . 

Transportation 

The guidelines from the USOE will control this and there will be 
some flexibility within the overall cost limits imposed by USOE . 

Food Service 

Food Services appear to be dependent upon the local resource. 



Clerical and Duplication Service 

Clerical and Duplication Service should be part of the built- in 
costs of the proposal. Adequacy and immediacy appear important. 
SmiL^y infoLtion sent to the participants can contribute to 

+-hp effectiveness of the seminar. 



other arrangements and services that may 
nF the seminar include such items as provision for weetend activi 

ties which should be related to the gMls of the 

able Field trips to industries may be valuable. A librery 

reso^rcrmaterials can be of service and help to participants. 
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The discussion indicated that people at the policy-making level 
were desirable for the PPBS groups. Adequate lead time to state 
directors for their nominations could aid in securing the people 
for whom the seminars could be of greatest value. Nominations could 
be solicited from several sources so that the desired spectrum was 
attained . 

The following suggestions may aid in planning and conducting 
Leadership Development Seminars: 

1. Objectives should be presented to participants in a succinct, 

clearly presented statement. 

2. Small group sessions may be used with ^^small problem 

assignments . '' 

3. Allow for questions following each major presentation. 

4. Schedule a ’’Brag and Lie” session to show what various 

states are doing. 

5. Increase the emphasis on techniques of working with groups. 

6. Provide opportunities for interim reports to be made during 

the conference . 

7. Allow adequate time for practice in the techniques of 

leadership. 

8. Include a session on practical politics and how to work 

within the political structure. 

9. Have ’’dress rehearsals” of major presentations. 

10. Do not compromise on the quality of presentations. 

11. Provide profiles of participants to the staff prior to 

the seminar. 

12. Provide advance information on the scope and objectives, 

etc., of the seminar to help participants prepare. 

13. Resource persons making presentations should provide 

advance copy for staff use. 



CONFERENCE EVALUATION 



LDS and PPBS Evaluation Conference 
University of Maryland, November 28, 1967 
Einar R. Ryden 



It was the purpose of this conference to weigh the strengths and 
weaknesses of the completed Leadership series and the Program Planning 
and Budgeting seminars. It was expected that constructive critique 
would lead to specific recommendations for future seminars. 



The conference began by demonstrations of how evaluation is an 
integral part of the teaching/learning/evaluation system and of the 
PPB System. For these to be effective evaluation is central, con- 
tinuous, integrated. It is the source of energy and through its effi 
cient use, improvement and further development surely can occur. 



Several illustrations were given and models shown as sources for 
stimulating thinking in the direction of evaluative 
gized by the systems approach. Evaluation is basic to all facets o 
a program and undergirds the whole. Evaluation is an ongoing process 
and it should be continuously employed to give direction to all 
activities. It is not merely an initial-terminal testing process . 
Implicit are, of collr^e, the initial and terminal evaluations of 
progress and success . 



The following two sections deal first with this conference, then 
with those aspects of this conference devoted to the planning of 
future seminars. 



This Conference 



Identification of Problem Areas in Past Seminars 

The leadership as well as the participants experienced fa^ to 
considerable difficulty in settling down to this tesk the first day. 
Constructive criticism should have been more readily available, since 
thf Srst responsibility dealt with strengths. It is a well known 
principle that one should discuss strengths before weaknesses. Some 
of ?hirdimculty was rectified later, but the purpose was not fully 

achieved . 

Invoivemenf of Parficipanfs 

Contributions in discussion ranged from ’Very much” to "occa- 
sionally.” All participants entered into discussion at some point. 
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’’Systems” Criteria 



One of the two groups applied the ’’systems” approach directly and 
proceeded to carry out its assignments in this manner. The other 
group devoted most of its time to stating objectives and scheduling. 



Planning Future Seminars 
Identifying Problems Areas for Future Seminars 

These were at times specifically stated and at other times could 
only be inferred in the discussion of objectives. In this sense j the 
lists were very long. Sometimes they were ’’task” oriented. 

Involvement 

A good deal of attention was given to providing participants 
with the opportunity to observe and practice. In finalizing a program, 
however, it will be important to examine carefully the portion of time 
the participants ’’listen” and the portion of time they will actively 
participate under planned supervision and guidance. 

’’Systems” Criteria 

To what extent were the ’’systems” concepts employed in planning? 
Some of the major concepts are: need, identification, objectives, 

alternatives, program, decisions, evaluation. 

Although objectives relating to the planning of a program were 
considered at length, there did not appear to be sufficient attention 
to the identification of terminal behavior, description of conditions 
under which the behavior will be expected to occur and specification 
of criteria for acceptable performance. Also, a number of statements 
of objectives did not, at least to this observer, communicate the 
instructional intent. 

What Remains to be Done? 

It will be necessary to evaluate and re-evaluate for the purpose 
of adjusting criteria and objectives. This involves the very im- 
portant concept: How to teach for the ob jecl.ives . 

I wish to raise three questions: 

1. If you insert the concepts of time and budget too early in 
the planning, isn’t there a risk that these may function as psycho- 
logical blocfe? 
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2. Are there varieties of planning, stages in the planning 
process and some planning groups concerned with budgeting only 
partially or not at all? 

3. Was sufficient attention given to making use of the new 
educational technology? It’s here now , you know. Let’s not Torget 
that one of the most significant contributions of learning theory 
today is reinforcement. If you make use of it, you will achieve 
many satisfying results of the teaching-learning process. 

Perhaps the major result of the conference was the effect of a 
program planning or ’’systems” approach upon the participants in group 
thinking. Evaluation, although only at times directly referred to, 
clearly permeated the discussion in many ways. This, after all, was 
the major goal and easily overshadows shortcomings here and there. 
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